Generally described, computing devices can be utilized in a variety of contexts such as for exchanging information, facilitating communication between users, facilitating the operation and control of a wide variety devices and processes, and the like. In the context of a manufacturing or production environment, a computing network made up of a number of computing devices, including personal computing devices, server computing devices, programmable logic controllers (PLCs), and/or other networked devices can be utilized in conjunction with a communication network, such as the Internet, to facilitate the operation and control of various devices/processes. For example, a networked PLC may be utilized to control the operation of physical manufacturing or processing equipment, such as controllers for valves, power supplies, pumps, machinery, etc. Similarly, a software application, or suite of software applications, may be hosted on a networked computing device (such as a server or personal computing device) to receive instructions regarding the operation of various equipment and transmit the appropriate respective instructions to the appropriate equipment (such as through a PLC).
A fault in one or more networked computing devices, such a fault in a computing device, can lead to the failure of associated equipment, loss of manufacturing/production time, property damage, and the like. Accordingly, manufacturing/production computing networks (including hardware and software aspects) can be designed with redundant components to avoid fault conditions during execution in a manufacturing/production environment. For example, a PLC may include a “fail safe” mode such that in the event of a fault, the outputs from the PLC mitigate potential damage to attached equipment or errant instructions that could cause additional faults/damage.
Computing systems can be tested to verify that requirements for safety and redundancy are met and to discover errors in design/implementation. For example, a testing system can be implemented, such as in a laboratory, that attempts to emulate various commands, instructions, or other environmental information and then measure the response generated by the computing device(s) being tested. The emulated commands, instructions, or other environment information can be embodied as a test case or testing procedure that can be executed by a testing system.
One approach to the generation of test cases and/or testing procedures for computing device testing involves the utilization of testing framework that is in communication solely with the device under test. This approach, however, can be inefficient in recreating many of the inputs and/or operating environments associated with the device under test. For example, in a system under test including multiple networked computing devices, the testing framework would need to analyze inputs/outputs from each computing device in the system under test in order to accurately test the system. For example, a computing device in a system under test experiencing an error may still be generating an expected output coinciding with an output generated by the computing device entering a “fail safe” mode. However, in such a fail safe mode, the computing device would not be able to properly process inputs from a control computing device. Accordingly, current testing approaches can become deficient in properly assessing system under test performance as a function of a testing environment.